Papers^γ

Question asked: how can you audit a recommender system if you don't have access to user-item interactions? Answer proposed: use a "recommendation graph" and simulate user behavior with a random walk with teleportation. Legitimate question, I am not completely convinced by the answer but it has some merits, in particular, simplicity. ### Introduction - to measure : diversity / segregation-polarization (not accuracy) - use the structure of an underlying network - 3 datasets, all movies RS : IMDB, Google Play, Netflix ; films have a genre in a list ##### Contributions : quantifying diversity - use a notion of graph: directed link from i to j if j is recommended on i page - measures based on films genre ; popularity based on the graph structure: in-degree, PageRank - segregation quantified with concentration and evenness (8: Chakraborti et al, 2017) ### Related work ##### diversity in RS - 9 : Nguyen et al, WWW 2014: CF impact on user behaviors - 10: Zhou et al, PNAS 2010 (Auralist) defines novelty - 11: Santini and Castells defines diversity et novelty with a fuzzy interpretation - 12: Vargas et Castells, RecSys 2011: novelty and diversity from a user perspective - 13: Lathia et al, SIGIR 2010: considers temporal aspects ##### on polarization dynamics - 20: DeGroot explanation from opinion modeling - 17: Dandekar et al, PNAS 2013 : polarizing effects of RS algorithms - two main schools of thought to explain polarization: either opinion is reinforced by interactions with like-minded people or people are exposed to opposite views then reject them (hence polarization ### Framework for auditing RS ##### Network construction : see above - possible to give weights to the links in tne network depending on item-item similarities, or based on the rank of j reco on i page - but overall different with literature: directed network and not based on similarity ##### User modeling : see above (RW with teleport), as we don't have access to navigation logs - then we consider distribution of types visited during the RW ##### Datasets : see above - general stats : see tab1 - collect by snowball sampling (from one node, then neighbors, then neighbors of neighbors... like a BFS) - to account for personalization effects, crawling from a unique IP adress - film genres: 15 for GP, 29 for Netflix and IMDB - compare distributions for 400 steps RW with no teleport (so, sort of random sampling): GP dominated by Action, IMDB more balanced ### Diversity in RS ##### existing measures - similarity between items computed with Jaccard coefficient => possible to compute usual measures (ILS (15), long-tail novelty (14), unexpectedness (23), source-list diversity(16)) - Table 2 : traditional measures on the datasets - observation : Netflix has greater source-list diversity, but IMDB more diverse according to other measures ##### new measures - assortativity by genre, by popularity (using as a proxy normalized centralities: in-degree, pagerank) => see Table III - contingency matrices: fraction of links from genre to genre (fig2); in general RS recommend from a genre to the same one + some specific relation between specific genres - equivalent for popularity: bins of popularity (bottom/middle/top) then count links from a bin to another (fig3); in general push toward long-tail, especially on IMDB ##### New measures based on RW - entropy of the genre distribution obtained - exploration when parameters vary (N = RW length, tp = teleportation probability, starting point) - fig4a : evol with t_p (small increase then plateau) - fig4b : évol with N (growth) - note that the RW is finite and we do not reach the steady state ##### Information segregation in RS * use measures in Chakraborty et al.(8) : evenness and concentration * evenness : to what extent a group is exposed uniformly to info units , it (1-Gini) with Gini computed on the genres consumed by the users of the group (1 ~ even consumption) * concentration : 1/2 . sum (fraction of films i consumed by the group).(fraction of the films consumed by the groups).(fraction of the films of genre i) ; if concentration is low what is consumed is close to what is proposed * results on fig5